1. Field of the Invention
This invention relates to a method and apparatus for extracting an abnormal pattern from an image. This invention particularly relates to an improvement in a morphology operation for extracting an image portion, which has a characteristic shape, from an original image represented by an original image signal.
2. Description of the Prior Art
Image processing, such as gradation processing or frequency processing, has heretofore been carried out on an original image signal, which represents an original image having been obtained with one of various image obtaining methods, such that a visible image having good image quality can be reproduced and used as an effective tool in, particularly, the accurate and efficient diagnosis of an illness. Particularly, in the field of medical images, such as radiation images of human bodies serving as objects, it is necessary for specialists, such as doctors, to make an accurate diagnosis of an illness or an injury of the patient in accordance with the obtained image. Therefore, it is essential to carry out the image processing in order that a visible image having good image quality can be reproduced and used as an effective tool in the accurate and efficient diagnosis of an illness.
As one of the image processing, there has heretofore been known the processing based upon the algorithm of morphology (hereinbelow referred to as the morphology operation or the morphology processing), with which only a specific image portion, such as an abnormal pattern, is selectively extracted from an original image.
The morphology processing has been studied as a technique efficient for detecting, particularly, a small calcified pattern, which is one of characteristic forms of mammary cancers. However, the image to be processed with the morphology processing is not limited to the small calcified pattern in a mammogram, and the morphology processing is applicable to any kind of image, in which the size and the shape of a specific image portion (i.e., an abnormal pattern, or the like) to be detected are known previously.
The morphology processing is carried out by using a structure element (also referred to as a mask) B corresponding to the size and the shape of the pattern to be extracted. The morphology processing has the features in that, for example, (1) it is efficient for extracting a calcified pattern itself, (2) it is not affected by complicated background information, and (3) the extracted calcified pattern does not become distorted.
Specifically, the morphology processing is advantageous over ordinary differentiation processing in that it can more accurately detect the geometrical information concerning the size, the shape, and the image density distribution of the calcified pattern.
How the morphology processing is carried out will be described hereinbelow by taking the detection of a small calcified pattern in a mammogram as an example. (Fundamental operation of morphology processing)
In general, the morphology processing is expanded as the theory of sets in an N-dimensional space. As an aid in facilitating the intuitive understanding, the morphology processing will be described hereinbelow with reference to a two-dimensional gray level image.
The gray level image is considered as a space, in which a point having coordinates (x, y) has a height corresponding to an image density value f(x, y). In this case, it is assumed that the image signal representing the image density value f(x, y) is a high luminance-high signal level type of image signal, in which a low image density (i.e., a high luminance when the image is displayed on a CRT display device) is represented by a high image signal level.
Firstly, as an aid in facilitating the explanation, a one-dimensional function f(x) corresponding to the cross-section of the two-dimensional gray level image is considered. It is assumed that a structure element g used in the morphology operation is a symmetric function of Formula (1) shown below, which is symmetric with respect to the origin. EQU g.sup.s (x)=g(-x) (1)
It is also assumed that the value is 0 in a domain of definition G, which is represented by Formula (2) shown below. EQU G={-m, -m+1, . . . , -1, 0, 1, . . . , m-1, m} (2)
In such cases, the fundamental forms of the morphology operation are very simple operations carried out with Formulas (3), (4), (5), and (6) shown below. EQU dilation; f.sym.G.sup.s !(i)=max {f(i-m), . . . , f(i), . . . , f(i+m)}(3) EQU erosion; f.crclbar.G.sup.s !(i)=min{f(i-m), . . . , f(i), . . . , f(i+m)}(4) EQU opening; f.sub.g =(f.crclbar.g.sup.s).sym.g (5) EQU closing; f.sup.g =(f.sym.g.sup.s).crclbar.g (6)
Specifically, as illustrated in FIG. 7A, the dilation processing is the processing for retrieving the maximum value in the region of a width of .+-.m (which width is the value determined in accordance with the structure element B and corresponds to the mask size shown in FIG. 7A) having its center at a picture element of interest. As illustrated in FIG. 7B, the erosion processing is the processing for retrieving the minimum value in the region of the width of .+-.m having its center at the picture element of interest. The opening processing is equivalent to the processing in which the dilation processing is carried out after the erosion processing, i.e., the processing in which the maximum value is searched after the searching of the minimum value. Also, the closing processing is equivalent to the processing in which the erosion processing is carried out after the dilation processing, i.e., the processing in which the minimum value is searched after the searching of the maximum value.
More specifically, as illustrated in FIG. 7C, the opening processing is equivalent to the processing for smoothing the image density curve f(x) from the low luminance side, and removing a convex image density fluctuating portion (i.e., the portion at which the luminance is higher than that of the surrounding portions), which fluctuates in a region spatially narrower than the mask size of 2 m.
Also, as illustrated in FIG. 7D, the closing processing is equivalent to the processing for smoothing the image density curve f(x) from the high luminance side, and removing a concave image density fluctuating portion (i.e., the portion at which the luminance is lower than that of the surrounding portions), which fluctuates in the region spatially narrower than the mask size of 2 m.
In cases where the structure element g is not symmetric with respect to the origin, the dilation operation with Formula (3) is referred to as the Minkowski sum, and the erosion operation with Formula (4) is referred to as the Minkowski difference.
In cases where the image signal representing the image density value f(x) is a high image density-high signal level type of image signal, in which a high image density is represented by a high image signal level, the relationship between the image density value f(x) and the image signal value becomes reverse to the relationship between the image density value f(x) and the image signal value in the high luminance-high image signal level type of image signal. Therefore, the dilation processing, which is carried out on the high image density-high signal level type of image signal, coincides with the erosion processing, which is carried out on the high luminance-high signal level type of image signal as shown in FIG. 7B. The erosion processing, which is carried out on the high image density-high signal level type of image signal, coincides with the dilation processing, which is carried out on the high luminance-high signal level type of image signal as shown in FIG. 7A. The opening processing, which is carried out on the high image density-high signal level type of image signal, coincides with the closing processing, which is carried out on the high luminance-high signal level type of image signal as shown in FIG. 7D. Also, the closing processing, which is carried out on the high image density-high signal level type of image signal, coincides with the opening processing, which is carried out on the high luminance-high signal level type of image signal as shown in FIG. 7C.
The morphology processing is herein described with respect to the high luminance-high signal level type of image signal.
(Application to detection of calcified patterns)
In order for a calcified pattern to be detected, it is considered to employ a difference method, in which a smoothed image signal is subtracted from the original image signal. However, with a simple smoothing method, it is difficult to discriminate the calcified pattern from an elongated non-calcified pattern (for example, a pattern of the mammary gland, a blood vessel, mammary gland supporting tissues, or the like). Therefore, Obata of Tokyo University of Agriculture and Technology, et al. have proposed a morphology filter, which is represented by Formula (7) shown below and is based upon the opening operation using a multiply structure element. Reference should be made to "Extraction of Small Calcified Patterns with A Morphology Filter Using A Multiply Structure Element," Collected Papers of The Institute of Electronics and Communication Engineers of Japan, D-II, Vol. J75-D-II, No. 7, pp. 1170-1176, July 1992; and "Fundamentals of Morphology and Its Application to Mammogram Processing," Medical Imaging Technology, Vol. 12, No. 1, January 1994.! ##EQU1##
In Formula (7), Bi (wherein i=1, 2, . . . , n) represents n number of linear structure elements, each of which has a size corresponding to the total size of m number of picture elements (in the example shown in FIG. 8, nine-picture element, four-direction structure elements are employed, and m=9, n=4). (The structure elements, as a whole, will hereinbelow be referred to as the m-picture element, n-direction multiply structure element.) In cases where the structure element Bi is set to be larger than the calcified pattern to be detected, a calcified pattern, which is a convex signal change portion finer than the structure element Bi (i.e., which is an image portion fluctuating in a spatially narrow region) and has luminance values larger than the luminance values of the surrounding portions, is removed in the opening processing. On the other hand, an elongated non-calcified pattern, such as a pattern of the mammary gland, is longer than the structure element Bi. Therefore, in cases where the inclination of the non-calcified pattern (i.e, the direction along which the non-calcified pattern extends) coincides with one of the directions of the four structure elements Bi, the non-calcified pattern remains unremoved after the opening processing, i.e. the operation of the second term of Formula (7), has been carried out. Therefore, when the smoothed image signal obtained from the opening processing (i.e. the signal representing the image, from which only the calcified pattern has been removed) is subtracted from the original image signal f, an image can be obtained which contains only the small calcified pattern. This is the concept behind Formula (7).
As described above, in cases where the image signal is of the high image density-high signal level type, the image density value of the calcified pattern is smaller than the image density values of the surrounding image portions, and the calcified pattern constitutes a concave signal change portion with respect to the surrounding portions. Therefore, the closing processing is applied in lieu of the opening processing, and Formula (8) shown below is applied in lieu of Formula (7). ##EQU2##
The closing processing carried out with Formula (8), which is an example of the morphology operation, will hereinbelow be described in detail.
Specifically, the morphology operation is carried out on the image density value Sorg, which is represented by the high image density-high signal level type of image signal. With the morphology operation, the maximum value processing (i.e., the dilation processing) is carried out on the image signal, which has a distribution of the image density value Sorg indicated by, for example, the solid line in FIG. 9A, by using a linear structure element B, which is constituted of three picture elements and is shown in FIG. 9B. As a result, an image density value S.sub.i of a certain picture element of interest is converted into S.sub.i ', which takes the maximum value S.sub.i+1 of the values of the three adjacent picture elements (determined by the structure element B) having their center at the picture element of interest. The operation is carried out for all of the picture elements constituting the image, each of them being taken as the picture element of interest. In this manner, the image signal having the distribution of the image density value Sorg indicated by the solid line in FIG. 9A is converted into the maximum value signal having the distribution of the image density value Sorg', which is indicated by the broken line in FIG. 9A.
Thereafter, the minimum value processing (i.e., the erosion processing) is carried out on the maximum value signal, which has been obtained from the maximum value processing, by using the structure element B. As a result, the maximum value signal S.sub.i ' corresponding to the picture element of interest indicated by the broken line in FIG. 9A is converted into S.sub.i " (=S.sub.i), which takes the minimum value S.sub.i-1 " of the values of the three adjacent picture elements having their center at the picture element of interest. The operation is carried out for all of the picture elements constituting the image, each of them being taken as the picture element of interest. In this manner, the minimum value signal Sorg" having the distribution indicated by the chained line in FIG. 9A is obtained from the minimum value processing. The image signal indicated by the chained line in FIG. 9A has the distribution such that the image portion corresponding to the signal change portion, at which the original image signal Sorg fluctuates in a spatially narrower range than the size of the structure element B, has been eliminated, and such that the image portion corresponding to the signal change portion, at which the original image signal Sorg fluctuates in a spatially wider range than the size of the structure element B, and the image portion, at which the original image signal Sorg does not fluctuates, are kept in the original forms. More specifically, the aforesaid processing (i.e., the closing processing) serves as the processing for smoothing the image density distribution from the high image density side.
The value having been obtained from the closing processing (i.e., the value having been obtained by carrying out the maximum value processing on the original image signal Sorg and then carrying out the minimum value processing) is subtracted from the original image signal Sorg, and a value Smor is thereby obtained. The thus obtained value Smor represents the image portion corresponding to the signal change portion, at which the signal value fluctuates in a spatially narrower range than the size of the structure element B and which has been eliminated by the aforesaid closing operation.
Fundamentally, an image signal represents spatial coordinates (x, y), which constitute a two-dimensional element, and a signal value f(x, y), which constitutes a third dimensional element. However, in the foregoing, as an aid in facilitating the understanding, the morphology operation is described with respect to the one-dimensional image signal distribution curve, which appears in a predetermined cross section of the image expanded in the two-dimensional plane.
Therefore, actually, it is necessary for the foregoing explanation to be applied to a two-dimensional image. Also, for the processing of a two-dimensional image, the multiply structure element is employed.
In the morphology operation represented by Formula (7) or Formula (8), with which the specific image portion is extracted from the image represented by the original image signal, it is necessary for the operation to be carried out in two steps, such that the minimum value processing may firstly be carried out on the original image signal and the maximum value processing may then be carried out, or such that the maximum value processing may firstly be carried out on the original image signal and the minimum value processing may then be carried out. Specifically, for example, in the closing processing, the maximum value processing is firstly carried out on the original image signal, and information representing the results of the maximum value processing is stored. Thereafter, the minimum value processing is carried out on the results of the maximum value processing.
In cases where the original image is constituted of an array of Y lines.times.X picture elements (the total number of picture elements being Y.times.X picture elements) and the closing processing is carried out by using the m-picture element, n-direction multiply structure element, it is necessary for the aforesaid maximum value processing to be carried out with respect to every picture element of the original image. Also, it is necessary for the maximum value processing to be carried out for each of the n number of structure elements. Therefore, as the information representing the results of the maximum value processing, which information is to be stored, Y.times.X.times.n pieces of information are obtained. For example, in cases where the original image is constituted of an array of 1,024 lines.times.1,024 picture elements and the number n of the directions of the structure elements constituting the multiply structure element is equal to eight, a storage space for approximately 8,400,000 (=1,024.times.1,024.times.8) pieces of information must be prepared in order to store the information representing the results of the maximum value processing.
The results of the maximum value processing are the intermediate ones for obtaining the results of the closing processing. It is not practicable to use such a wide storage space for approximately 8,400,000 pieces of information for the storage of the information representing the intermediate results.